EA 002 062 



ED 027 622 

By-Berty, Ernest 

Some Principles and Practices of Evaluation. 

Pub Date 1 Nov 68 

Note* 19p.; Paper presented to the West Virginia State Department of Education In-Service Program 
(November 1. 1968). 

EDRS Price MF-S0.25 HC*$1.05 

Descriptors- Behavioral Objectives* Bibliographies* * Educational Objectives* Educational Programs, * Evaluation 
Methods, * Program Evaluation 

Evaluation is the process of determining the extent to which specified objectives 
have been reached. As ' a professional tool, it is used to (1) appraise the 
achievements of individual students; (2) diagnose the learning problems of an 
individual student or class in order to devise future teaching strategies; and (3) 
appraise the effectiveness of curricula, courses, instructional materials, and 
administrative and organizational structures. Product evaluation, measurement of the 
performance of students, holds the greatest promise for helping to improve the 
overall educational program. Steps in product evaluation include (1) definition of 
educational objectives to be achieved. (2) translation of the goals into descriptions 
of behavior. (3) identification of situations in which the behavior can be observed. (4) 
development of some measurement device. (5) application of the evaluation device to 
the educational program, (6) analysis of evidence obtained, and (7) statement of 
conclusions. Once evaluation has been expanded into performance terms, difficult 
tasks of data collection, organization, analysis, and interpretation arise. A 
bibliography listing 87 books, pamphlets, and articles is included to assist in further 
study of evaluation. (TT) 
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SOME PRINCIPLES AND PRACTICES OF EVALUATION 



THE TITLE OF MY PAPER IS "SOME PRINCIPLES AND PRACTICES OF EVALUATION." ANY 
OF YOU AT ALL FAMILIAR WITH THIS TOPIC KNOW VERY WELL IT COULD COVER SEVERAL 2-3- 
HOUR CREDIT COURSES. OBVIOUSLY, THE TIME ALLOTTED MAKES IT MANDATORY THAT THE 
TOPIC BE DELIMITED. IN THE DEVELOPMENT OF THIS PAPER I WAS ALSO FACED WITH TWO 
OTHER CHARGES: 

A. MAKE IT RELEVANT TO THE FOLLOWING ACTIVITY, AND 

B. USE THE KISS TECHNI QUE— KEEP IT SIMPLE STUPID’ 

I HAVE TRIED TO ABIDE WITH ALL THREE CHARGES— THE TIME ELEMENT, THE RELEVANCY, 
AND THE SIMPLENESS. 

MY PAPER HAS BEEN ORGANIZED AROUND A SERIES OF QUESTIONS, SUCH AS: 

1. WHY EVALUA T E EDUCATIONAL PROGRAMS? 

2. WHAT IS EVALUATION? 

3. WHAT METHOD OF EVALUATION SHOULD BE APPLIED? 

4. HOW DO YOU PLAN FOR THE SO-CALLED "PRODUCT TYPE" OF EVALUATION? 

5. WHEN SHOULD EVALUATION BE ATTEMPTED? 

6. WHAT IS THE PRESENT TREND IN EVALUATION? 

BEFORE I BEGIN TO ANSWER ANY OF THE QUESTIONS I WOULD LIKE TO REFER TO SOME 
TERMINOLOGY USUALLY ASSOCIATED WITH EVALUATION AND ATTEMPT TO APPLY SOME GROUND 
RULES. TERMS SUCH AS EVALUATION, ASSESSMENT, APPRAISAL, AND MEASUREMENT, ALL MEAN 
TO MEASURE SOMETHING . THUS, AS FAR AS I AM CONCERNED THEY CAN BE USED INTERCHANGE- 
ABLY. I'M PRETTY SAFE IN MAKING THIS STATEMENT BECAUSE THIS IS THE WAY THEY ARE 
USED IN THE LITERATURE. THE WORD SURVEY ALSO APPEARS TO BE USED BY SOME AUTHORS 
TO MEAN ABOUT THE SAME THING; HOWEVER, IT SIGNIFIES SOMETHING A LITTLE DIFFERENT 
TO ME. I USUALLY LIKE TO THINK OF A SURVEY BEING USED WHEN A PROJECT IS FIRST 
PLANNED. IT ASSISTS IN DETERMINING THE FEASIBILITY OF A PROPOSED PROGRAM BY 
DESCRIBING THE ENVIRONMENT AND IDENTIFYING SOME OF THE UNMET NEEDS. 
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THERE ARE OTHER TERMS I PROBABLY SHOULD ALSO DEFINE SUCH AS BEHAVIORAL OB- 
JECTIVES, LEARNING OUTCOMES, PERFORMANCE TERMS AND OPERATIONAL TERMS; HOWEVER, THE 
ELEMENT OF TIME DOESN'T PERMIT US TO LINGER VERY LONG ON DEFINITIONS. WE MUST 
NOW MOVE INTO ANSWERING THE QUESTIONS RAISED. 

WHY EVALUATE EDUCATIONAL PROGRAMS? 

FOR MANY REASONS, EVALUATION HAS SUDDENLY ACHIEVED A NEW PROMINENCE. THE 
EDUCATION COMMUNITY AND THE LAY PUBLIC AS WELL HAVE SUDDENLY COME TO BELIEVE THAT 
EVALUATION IS NOT ONLY POSSIBLE, BUT IS ESSENTIAL. AS A MATTER OF FACT, WE HAVE 
BEEN TOLD THIS FOR YEARS, NOW THEY ARE LOOKING FOR US TO DELIVER. THIS PRESENTS 
US WITH AN UNPARALLELED OPPORTUNITY, BUT IT ALSO PRESENTS US WITH AN EXTRA- 
ORDINARILY MESSY SET OF PROBLEMS IN HOW TO GET IT DONE. THE JOB IS MUCH TOUGHER 
THAN MOST PEOPLE REALIZE. 

COMMISSIONER OF EDUCATION, HAROLD HOWE, CITES ONE OF THE SIGNIFICANT REASONS 
IN THE FOLLOWING QUOTATION: 

...EVALUATION HAS BECOME A SORT OF SACRED COW IN EDUCATION IN 
THE SENSE THAT EVERYONE, BOTH EDUCATORS AND LAYMEN, THEORETICALLY 
BELIEVE IN IT. NOT MANY PEOPLE KNOW WHAT IT IS OR WHAT TO DO 
WITH IT, BUT IT IS NEVERTHELESS WIDELY REGARDED AS 'A GOOD THING.' 

THE VAGUE NOTION IS THAT WE CAN FIND OUT THROUGH SOME PROCEDURE 
(SCIENTIFIC OR OTHERWIDE) WHAT WE ARE ACCOMPLISHING WITH ALL THESE 
FEDERAL DOLLARS SPENT ON EDUCATION. 

BESIDES THE NEED TO SATISFY THE LEGAL REQUIREMENTS OF THE VARIOUS FEDERAL 
PROGRAMS AS WELL AS PUBLIC OPINION WHICH DETERMINES THE MORAL AND FINANCIAL 
SUPPORT GIVEN TO EDUCATIONAL POLICIES AND PRACTICES — EVALUATION IS A PRO- 
FESSIONAL TOOL WHICH SHOULD PROVIDE EDUCATORS WITH SOUND AND ADEQUATE 
INFORMATION ON EDUCATIONAL RESULTS. 
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AS A PROFESSIONAL TOOL, EVALUATION IS USED: 

1. TO APPRAISE THE ACHIEVEMENTS OF INDIVIDUAL STUDENTS. 

2. TO DIAGNOSE THE LEARNING DIFFICULTIES OF AN INDIVIDUAL STUDENT OR CLASS 
IN ORDER TO DEVISE FUTURE TEACHING STRATEGIES. 

3. TO APPRAISE THE EFFECTIVENESS OF A CURRICULUM, OF A COURSE, OF INSTRUC- 
TIONAL MATERIALS AND PROCEDURES, AND OF ADMINISTRATIVE AND ORGANIZATIONAL 
STRUCTURES AND PROCESSES. 

IN SUMMARY, PROFESSIONALS USE EVALUATION AS A MEANS OF ENCOURAGING PROGRAM 
MODIFICATION AND REVISION, OR EVEN THE ABANDONMENT OF THE PROGRAM ALTOGE’t IER, 

IT CAN BE USED TO DETERMINE THE EFFECTIVENESS OF A PROGRAM OR PROJECT. 

SINCE THE STATE DEPARTMENT USUALLY HAS THE RESPONSIBILITY OF COMBININL THE 
FINAL RESULTS OF PROJECTS AND PROGRAMS UNDER ITS AUSPICES, IT IS NECESSARY THAT 
THIS GROUP ADVOCATE EVALUATION PROCEDURES WHICH WILL GREATLY FACILITATE THE 
ASSESSMENT OF THE OVERALL IMPACT. 

WHAT IS EVALUATION? 

AS USED IN THIS PAPER, EVALUATION IS THE PROCESS OF DETERMINING THE EXTENT 
TO WHICH SPECIFIED OBJECTIVES HAVE BEEN REACHED. STATED IN ANOTHER WAY, EVALU- 
ATION IS THE PROCESS OF ASSESSING THE EXTENT AND DIRECTION RESULTING FROM AN 
EDUCATIONAL EXPERIENCE. OBVIOUSLY, 3ASELINE DATA IS REQUIRED TO MAKE THOSE 
DECISIONS WHICH DETERMINE THE DIRECTION OF THE CHANGE PROCESS. EVALUATION 
SHOULD ANSWER THIS QUESTION - HAVE THE EDUCATIONAL ATTAINMENTS OF CHILDREN 
PARTICIPATING IN THE PROGRAM BEEN RAISED? 

RALPH TYLER, CHAIRMAN OF THE EXPLORATION COMMITTEE OF THE NATIONAL ASSESS- 
MENT PROJECT STATED: 

... "EVALUATION IS THE PROCESS FOR DETERMINING THE DEGREE TO 




WHICH CHANGES IN BEHAVIOR ARE ACTUALLY TAKING PLACE/ 1 



if 



ROBERT TRAVERS IN HIS BOOK EDUCATIONAL MEASUREMENT POINTS OUT THAT EVALUATION 
IS MORE THAN MERELY DETERMINING THE OUTCOMES (I.E., CHANGES PRODUCED IN THE 
STUDENTS). HE STATES THAT EVALUATION ALSO INVOLVES A JUDGMENT OF THE DESIRABILITY 
OF WHATEVER OUTCOMES ARE DEMONSTRATED TO OCCUR. 

WHAT METHOD OF EVALUATION SHOULD BE APPLIED? 

THIS PAPER IS BASED UPON THE PREMISE THAT THE KIND OF EVALUATION WHICH HOLDS 
THE GREATEST PROMISE FOR HELPING TO IMPROVE THE OVERALL EDUCATIONAL PROGRAM IS 
PRODUCT EVALUATION . THIS APPROACH PRESENTS THE PROBLEM OF DEVISING PROCEDURES 
WHICH WILL YIELD TRUTHFUL AND ACCURATE APPRAISALS OF PRODUCTS THAT REPRESENT 
STUDENT OUTCOMES. 

NOW I HAVE RAISED THE QUESTION IN YOUR MIND - JUST WHAT IS PRODUCT EVALUATION ? 

IN FORMULATING EDUCATIONAL OBJECTIVES, EDUCATORS HAVE DRAWN THEM FROM THREE 
DISTINCT DOMAINS, WHICH MAY BE DESIGNATED AS: 

1. STRUCTURE OUTCOMES 

2. PROCESS OUTCOMES 

3. PRODUCT OUTCOMES 

LET ME TRY TO DEFINE EACH OF THESE DOMAINS FOR YOU. 

1. STRUCTURAL OUTCOME - REFERS TO SOME ASPECT OF SCHOOL PLANT, EQUIPMENT, 

OR ADMINISTRATIVE ORGANIZATION. EXAMPLES MAY BE STUDENT-TEACHER RATIOS, 
NUMBER OF CLASSROOMS, TEACHER SALARIES, TOTAL ENROLLMENT, AGE OF 
FACILITIES, ETC. 

2. PROCESS OUTCOME - REFERS TO SOME ASPECT OF SCHOOL OR CLASSROOM PROCEDURE. 

EXAMPLES MAY BE: GUIDANCE COUNSELORS IN BOTH THE ELEMENTARY AND SECONDARY 

SCHOOLS, MODERN MATH, I.T.A., OPEN-COURT, ETC. 
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EVALUATION OF BOTH THE STRUCTURE AND PROCESS OUTCOME TYPES, GENERALLY CAN ONLY 
ASSUME A HIGH CORRELATION EXISTS WITH DESIRED OUTCOMES. THE INSTRUMENTS USED TO 
EVALUATE OBJECTIVES IN THESE DOMAINS TAKE THE FORM OF CHECKLISTS AND RATING SCALES. 
THESE ARE THE SO-CALLED “DO-IT-YOURSELF" EVALUATION DEVICES, SUCH AS: 

A. EVALUATIVE CRITERIA OF THE COOPERATIVE STUDY OF SECONDARY SCHOOLS 

B. CRITERIA FOR EVALUATING JUNIOR HIGH SCHOOLS 

C. OTHERS 

SUCH APPRAISALS INVOLVE A SERIES OF ASSUMPTIONS THAT CERTAIN STRUCTURES OR PRO- 
CESSES WILL IN FACT PRODUCE THE DESIRED END PRODUCT, ASSUMPTIONS WHICH ARE RARELY 
SUBJECTED TO TESTS. 

NOW, WHAT IS PRODUCT EVALUATION? IT REFERS TO SOME PERFORMANCE EXHIBITED BY 
A STUDENT THROUGH WHICH HE SHOWS HIS ABILITY, INTEREST, ATTITUDE, OR ADJUSTMENT 
(I.E., WHEN THE STUDENT SPONTANEOUSLY ENGAGES IN A WIDE RANGE OF GOOD READING). 

BECAUSE OF THE DIFFICULTIES IN MEASURING THE PRODUCTS OF EDUCATION, WHICH 
REPRESENT THE ULTIMATE OBJECTIVES OF THE EDUCATIONAL ENTERPRISE, THERE IS A 
STRONG TENDENCY BY EDUCATORS TO RETREAT TO APPRAISING THE STRUCTURE OF THE SCHOOL 
SYSTEM AND THE PROCESSES GOING ON WITHIN. NEVERTHELESS, IT IS THE PRODUCT OUTCOME 
TYPE OF EVALUATION WHICH WILL BE DISCUSSED FURTHER IN THIS PAPER. 

HOW DO YOU PLAN FOR THE PRODUCT TYPE OF EVALUATION? 

PROBABLY THE FIRST THING WE SHOULD DO IS TAKE A LOOK AT THE NECESSARY STEPS 
IN THE EVALUATION PROCESS, AND THEN GO BACK AND FOCUS ON ONE OR TWO OF THE MOST 
DIFFICULT STEPS AS TIME WILL PERMIT. 

HOWEVER, BEFORE WE GET INTO THE “STEPS," I WOULD LIKE TO EMPHASIZE AT LEAST 
TWO POINTS. FIRST, THE DEVELOPMENT OF AN EVALUATION PLAN RELATIVE TO ANY EDUCA- 
TIONAL PROGRAM EITHER BEGINS WHEN YOU FIRST BEGIN TO THINK ABOUT THE PROGRAM OR 
EVEN BEFORE. (I.E., YOU MAY WANT TO CONDUCT A FEASIBILITY STUDY TO IDENTIFY THE 
UNMET NEEDS.) 
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THE SECOND POINT DEALS WITH OBJECTIVITY. AS WE THINK OF EVALUATION, WE SHOULD 
NOT ATTEMPT TO PROVE ANY PARTICULAR POINT OF VIEW. OUR EVALUATION PROCEDURES MUST 
BE CARRIED OUT HONESTLY AND W>TH NO PRECONCEIVED NOTIONS OF EXACTLY WHAT THE END 
RESULT OUGHT TO BE OR MUST BE. ENOUGH OF THAT! NOW, LET'S TAKE A LOOK AT THE 
VARIOUS STEPS ONE GOES THROUGH IN USING THE PRODUCT OUTCOME TYPE OF EVALUATION. 

STEP 1. DEFINE EDUCATIONAL OBJECTIVES (PREFERABLY IN TERMS OF SPECIFIC 
UNITS OF OUTPUT) TO BE ACHIEVED THROUGH THE EXPERIENCES BEING 
EVALUATED. 

STEP 2. TRANSLATION OF THE EDUCATIONAL GOALS INTO DESCRIPTIONS OF 

BEHAVIOR WHICH WILL BE DISPLAYED IF THE OBJECTIVES ARE. ACHIEVED. 

STEP 3. IDENTIFICATION OF SITUATIONS IN WHICH THE PRESENCE OR ABSENCE 
OF THE DESIGNATED BEHAVIOR CAN BE OBSERVED AND RECORDED. 

STEP 4. ESTABLISHMENT OF SOME TYPE OF INTERPRETIVE DEVICE (STANDARD 
OR NORM) WHICH CAN BE USED IN MEASURING DESIRED GROWTH. 

STEP 5. APPLICATION OF THE EVALUATION METHODS DERIVED FROM STEPS 
3 AND 4 TO ALL THOSE PARTICIPATING IN THE PROGRAM. 

STEP 6. ANALYSIS OF EVIDENCE OBTAINED BY THE EVALUATION DEVICES IN TERMS 
OF PROGRESS TOWARD THE DEFINED OBJECTIVES. 

STEP 7. STATEMENT OF CONCLUSIONS REGARDING EFFECTIVENESS IN TERMS OF 
THE EXTENT TO WHICH OBJECTIVES WERE ACHIEVED. 

AT THIS TIME, I WOULD LIKE TO REFER BACK TO STEPS I AND 2: DEFINING GOALS 

AND TRANSLATING THEM INTO DESCRIPTIONS OF BEHAVIOR. THE PROCESS OF ENUMERATING 
THE GOALS OF THE PROPOSED PROGRAM IS PRIMARILY A RATIONALE AND JUDGMENTAL MATTER. 
JUDGMENTS MAY BE MADE BY VARIOUS GROUPS AND GATHERED BY VARIOUS MEANS. (I.E., 
CONFERENCE AND COMMITTEE DISCUSSION.) 
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AS A RULE, THE INITIAL GOALS WHICH EMERGE DO NOT PROVIDE A SUITABLE BASIS FOR 
A PROGRAM OF EVALUATION, SINCE THEY ARE USUALLY EXPRESSED IN VERY GENERAL TERMS. 
GOALS EXPRESSED IN GENERAL TERMS FREQUENTLY ARE VAGUE, CONVEY DIFFERENT MEANINGS 
TO DIFFERENT READERS, AND ARE FAR REMOVED FROM THE PRACTICAL OPERATION EITHER OF 
TEACHING OR APPRAISING. 

THE DIFFICULT TASK IS THAT OF TRANSLATING BROAD, VAGUE GOALS INTO MORE SPECIFIC 
OBJECTIVES OR OUTCOMES. THE FOLLOWING QUALITIES ARE NEEDED IF THE OBJECTIVES ARE TO 
PROVIDE A USEFUL FRAMEWbRK FOR APPRAISING THE RESULTS OF ANY EDUCATIONAL EXPERIENCE 

A. THE SPECIFIC OBJECTIVES ARE AGREED UPON AS REPRESENTING THE GENERAL 
GOALS. 

B. THE SPECIFIC OBJECTIVES HAVE COMMON MEANING TO ALL READERS. 

C. THE SPECIFIC OBJECTIVES ARE CLOSELY RELATED TO STUDENT BEHAVIOR. 

THE GREATEST EXAMPLE OF THE PRODUCT EVALUATION APPROACH IS THE NATIONAL EDU- 
CATIONAL ASSESSMENT PROJECT WHICH WAS UNDER THE DIRECTION OF RALPH TYLER. AS YOU 
PROBABLY KNOW, INSTRUMENTS ARE BEING OR HAVE BEEN CONSTRUCTED BY FOUR LEADING TEST 
DEVELOPMENT AGENCIES IN THE FIELDS OF: 

A. READING AND THE LANGUAGE ARTS E. CITIZENSHIP 

B. SCIENCE F. FINE ARTS 

C. MATHEMATICS G. VOCATIONAL EDUCATION 

D. SOCIAL STUDIES 

LET'S JUST FOCUS ON THE ASSESSMENT IN THE FIELD OF SCIENCE FOR THE PURPOSE 
OF THIS PAPER. THE DELINEATION OF THE MAJOR OBJECTIVES INTO PERFORMANCE TERMS 
HELPS TO ILLUSTRATE HOW IT CAN BE DONE. OF COURSE, THE WHOLE PROJECT IS AIMED 
AT REFLECTING THE CONTRIBUTIONS OF SCIENCE TO THE EDUCATION OF STUDENTS WHICH THE 
SCHOOLS ARE SERIOUSLY SEEKING TO ATTAIN. FOUR MAJOR GOALS OF SCIENCE EDUCATION 
WERE ARRIVED AT BY A DISTINGUISHED PANEL OF NATIONAL LEADERS IN THE FIELD. 
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THESE FOUR GOALS ARE THAT STUDENTS SHOULD COME TO: 

I. KNOW THE FUNDAMENTAL FACTS AND PRINCIPLES OF SCIENCE. 

II. POSSESS THE ABILITIES AND SKILLS NEEDED TO ENGAGE IN THE 
PROCESS OF SCIENCE. 

III. UNDERSTAND THE INVESTIGATIVE NATURE OF SCIENCE. 

IV. IiAVE ATTITUDES ABOUT AND APPRECIATIONS OF SCIENTISTS, SCIENCE, AND 
THE CONSEQUENCES OF SCIENCE THAT STEM FROM ADEQUATE UNDERSTANDING. 

LET US SEE, FOR INSTANCE, HOW GOAL II WAS TRANSLATED INTO BEHAVIORAL TERMS. 
NOTICE THEY HAVE IDENTIFIED FOR DEMONSTRATION, TEN ABILITIES AND SKILLS NECESSARY 
TO ENGAGE IN THE PROCESSES OF SCIENCE. 

GOAL II. POSSESS THE ABILITIES AND SKILLS NEEDED TO ENGAGE IN THE PROCESS 
OF SCIENCE . 

1. ABILITY TO IDENTIFY AND DEFINE A SCIENTIFIC PROBLEM. 

2. ABILITY TO SUGGEST OR RECOGNIZE A SCIENTIFIC HYPOTHESIS. 

3. ABILITY TO PROPOSE OR SELECT VALIDATING PROCEDURES, BOTH LOGICAL AND 
EMPIRICAL. 

A. ABILITY TO OBTAIN REQUISITE DATA. 

5. ABILITY TO INTERPRET DATA. 

6. ABILITY TO CHECK THE LOGICAL CONSISTENCY OF HYPOTHESES WITH RELEVANT 
LAWS, FACTS, OBSERVATIONS, OR EXPERIMENTS. 

7. ABILITY TO REASON QUANTITATIVELY AND SYMBOLICALLY. 

8. ABILITY TO DISTINGUISH AMONG FACT, HYPOTHESIS, AND OPINION, THE RELEVANT 
FROM THE IRRELEVANT, AND THE MODEL FROM THE OBSERVATIONS THE MODEL WAS 
DEVISED TO DESCRIBE. 

9. ABILITY TO READ SCIENTIFIC MATERIALS CRITICALLY. 

10. ABILITY TO EMPLOY SCIENTIFIC LAWS AND PRINCIPLES IN FAMILIAR OR UNFAMILIAR 
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WHAT DESIRABLE BEHAVIOR DID THEY ENUMERATE RELATIVE TO THE GOAL DEALING WITH 
ATTITUDES AND APPRECIATION? WELL, LET'S TAKE A LOOK. 

GOAL IV. HAVE ATTITUDES ABOUT AND APPRECIATIONS OF SCIENTISTS. SCIENCE, AND 

l 

THE CONSEQUENCES OF SCIENCE THAT STEM FROM ADEQUATE UNDERSTANDING . 

1. RECOGNIZE THE DISTINCTION BETWEEN SCIENCE AND ITS APPLICATIONS. 

2. HAVE ACCURATE ATTITUDES ABOUT SCIENTISTS. 

3. UNDERSTAND THE RELATIONSHIP BETWEEN SCIENCE AND MISCONCEPTIONS OR 
SUPERSTITIONS. 

4. BE READY AND WILLING KNOWINGLY TO APPLY AND UTILIZE BASIC SCIENTIFIC 
PRINCIPLES AND APPROACHES, WHERE APPROPRIATE IN EVERYDAY LIFE. 

5. BE INDEPENDENTLY CURIOUS ABOUT AND PARTICIPATE IN SCIENTIFIC 
ACTIVITIES. 

THE WHOLE POINT OF THE REFERENCE TO THE NATIONAL EDUCATIONAL ASSESSMENT 
PROJECT IS TO DEMONSTRATE HOW SOME PEOPLE HAVE TRANSLATED OR EXPANDED GOALS INTO 
A DEFINITE CHANGE RESULTING FROM AN EDUCATIONAL EXPERIENCE. 

I AM NOT SUGGESTING THIS IS AN EASY TASK, BUT I AM SUGGESTING IT CAN BE DONE. 
WE CANNOT SIT BACK AND LET A FEW PEOPLE IN THE GRADUATE SCHOOLS AND UNIVERSITIES 
DO IT FOR US. NEITHER CAN WE DISREGARD OUR PROFESSIONAL RESPONSIBILITY BY SAYING 
WE DO NOT HAVE THE INSTRUMENTS TO MEASURE ALL OF THE QUALITATIVE CHANGES IN BE- 
HAVIOR WHICH MAY OCCUR AMONG STUDENTS. THE ONLY WAY WE ARE GOING TO CRACK THIS 
"NUT" IS TO TRY IT. 



WHEN SHOULD EVALUATION BE ATTEMPTED? 

THERE ARE MANY SITUATIONS IN EDUCATION IN WHICH EVALUATION CAN BE USED. 
HOWEVER, I THINK THE MOST APPROPRIATE USE, FROM OUR POINT OF VIEW, IS ITS 
APPLICATION TO THE SO-CALLED PILOT PROGRAM. AS YOU WELL KNOW, THE PURPOSE OF 
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A PILOT PROGRAM IS TO DETERMINE WHETHER AN ACTIVITY WILL LEAD TO INCREASED KNOW- 
LEDGE AND/OR SKILLS FOR THOSE EXPOSED TO THE PROGRAM. IF THOSE STUDENTS EXPOSED 
SHOW SIGNIFICANT GAINS AS A RESULT OF THE PROGRAM, THEN IT CAN BE INCORPORATED 
INTO THE OVERALL CURRICULUM ON A PERMANENT BASIS. IT ENABLES US TO TEST A 
PARTICULAR APPORACH WITHOUT MAKING A FULL-SCALE COMMITMENT. 

THE PURPOSE OF ANY PILOT PROGRAM !S DEFEATED IF IT IS NOT ADEQUATELY EVALUATED. 
THE RESULT, OR OUTCOME, OF A PILOT PROGRAM MUST BE EVALUATED IN TERMS OF SPECIFIC 
CRITERIA BEFORE ONE CAN BE ASSURED OF ITS PERMANENT INCLUSION IN THE CURRICULUM. 

THUS, EVALUATION MUST BE AN INTEGRAL PART OF A PILOT PROGRAM. IN DESIGNING 
THE PROGRAM, EVALUATION NEEDS TO BE DESIGNED IN. IT SHOULD NOT COME AS AN AFTER- 
THOUGHT TO AN ALREADY EXISTING PROGRAM, BUT AS A BASIC PART OF THE PROGRAM ITSELF. 

WHAT IS THE PRESENT TREND IN EVALUATION? 

THE WAY I WOULD LIKE TO ANSWER THIS QUESTION IS TO VERY BRIEFLY ENUMERATE SOME 
OF THE HIGH LIGHTS IN THE HISTORY OF EVALUATION. 

EVALUATION OF EDUCATION HAS A HISTORY AS LONG AS THAT OF EDUCATION ITSELF. 

1. APPRAISING HUMAN BEHAVIOR BECAME COMMON ONLY IN THE LATTER PART OF 
THE 19TH CENTURY. , 

2. ACCREDITATION OF SECONDARY SCHOOLS BY REGIONAL GROUPS OF COLLEGES 
BEGAN ABOUT 1900. 

3. TEST AND MEASUREMENT MOVEMENT RECEIVED GREAT IMPETUS DURING 1920 *S. 

4. A PENNSYLVANIA STUDY CONDUCTED ABOUT 1928-38 BY WOOD AND LEARNED SET OUT 
TO REVIEW THE SYSTEM OF SCHOOLING IN THE LIGHT OF OBJECTIVE EVIDENCE. 

THE PROJECT SEEMED TO MEASURE SUPERFICIAL KNOWLEDGE OF FACTS RATHER THAN 
DEEP KNOWLEDGE AND THOUGHT PROCESSES. WOOD LATER ESTABLISHED THE 
COOPERATIVE TEST SERVICE, WHILE LEARNED ESTABLISHED THE GRADUATE RECORD 
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5. SINCE WORLD WAR II THE FOCUS OF INTEREST AND EFFORT APPEARS TO HAVE 
SHIFTED AWAY FROM A RESEARCH ORIENTED ATTEMPT TO DEVELOP NEW AND BETTER 
EVALUATION PROCEDURES TO AN ACTION RESEARCH ORIENTED TO INVOLVE SCHOOL 
PERSONNEL IN EVALUATING THEIR OWN PROGRAMS. THIS TYPE OF SELF EVALUATION 
HAS BEEN DEFENDED ON THE GROUND THAT THE EVALUATION EXPERIENCE WILL BE 

A VALUABLE LEARNING EXPERIENCE FOR THE TEACHERS WHO ENGAGE IN IT. 

6. RECENT LITERATURE, HOWEVER, APPEARS TO INDICATE A STRONG RESURGENCE OF 
THE RESEARCH ORIENTATION TYPE OF EVALUATION WITH EMPHASIS ON THE PRODUCT. 

7. APPEARING ON THE HORIZON IS THE CHALLENGE OF PROGRAM ANALYSIS OR SYSTEM 
ANALYSIS. THIS TYPE OF APPROACH HAS BEEN GIVEN IMPETUS BY ITS SUCCESSFUL 
USE IN INDUSTRY AND THE AVAILABILITY OF TOOLS, SUCH AS COMPUTERS. 

SYSTEMS ANALYSIS IS A COST-BENEFITED MANAGEMENT TYPE OF PROCESS. AT- 
TACHING NEW PROGRAMS, ESPECIALLY THE FEDERAL PROGRAMS, TO THE ALREADY 
EXISTING SCHOOL ENTERPRISE IS LIKE HANGING CHRISTMAS BULBS ON A 
CHRISTMAS TREE. IT BECOMES VERY DIFFICULT TU .lEASURE THE EFFECT OF 

ANY ONE OF THESE ATTACHED PROGRAMS ON THE EXISTING EDUCATIONAL ENTER- 
PRISE. SYSTEMS ANALYSIS HOLDS PROMISE OF DOING THIS JOB. HOWEVER, IN 
ITS EMBRYOTIC STAGE IT TOO SEEMS TO BE CONCENTRATING ON STRUCTURES AND 
PROCESSES. WHAT IS EVENTUALLY HOPED FOR IS AN INTERFACING OF THE THREE 
DOMAINS: STRUCTURE, PROCESS, AND PRODUCT. 
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CONCLUSION 

AS I SAID IN THE BEGINNING, EVALUATION IS A VERY BROAD TOPIC AND IT'S 
DIFFICULT TO COVER ALL OF ITS RAMIFICATIONS IN SUCH A SHORT LENGTH OF TIME. 

I HAD TRIED TO FOCUS ON THOSE AREAS WHICH I THOUGHT WOULD BE MOST PERTINENT 
TO YOUR WORK TODAY. I HAVE ATTEMPTED TO PIN-POINT SOME OF THE PRINCIPLES 
AND PRACTICES WHICH I THOUGHT WOULD BE OF THE GREATEST VALUE TO YOU. I 
TRIED TO CENTER ON THE TRANSLATION OR EXPANSION OF GOALS INTO PERFORMANCE 
TERMS AS THEY RELATE TO THE "PRODUCT TYPE" EVALUATION. HOWEVER, YOU MUST 
REMEMBER THERE ARE SEVERAL OTHER STEPS IN THE EVALUATION PROCESS WHICH WERE 
NOT TREATED AT ALL. THESE STEPS CAN ALSO BECOME QUITE STICKY. ONCE YOU 
HAVE EXPANDED YOUR GOALS INTO PERFORMANCE TERMS, THEN YOU WILL BE FACED 
WITH THE FOLLOWING TASKS: 

A. THE SELECTION OR DEVELOPMENT OF INSTRUMENTS OR TESTS TO COLLECT 
YOUR DATA. 

B. THE ORGANIZATION OF YOUR DATA. 

C. THE ANALYSIS OF YOUR DATA. 

D. THE INTERPRETATION OF THE RESULTS. 

UNDOUBTEDLY, SOME OF THESE TASKS WILL PROBABLY CALL FOR SPECIAL COM- 
PETENCIES WHICH NOT ALL OF US POSSESS, FOR THE BENEFIT OF THOSE OF YOU WHO 
WOULD LIKE TO PURSUE THIS TOPIC OF EVALUATION AT GREATER LENGTH, I HAVE 
PREPARED A BIBLIOGRAPHY TO DISTRIBUTE AMONG YOU. IT CAN BE USED AS A GUIDE 




POST IN YOUR READING 
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**Data obtained from individual school reports to the Mississippi Accrediting Commission, and the 
State Accrediting Commission, 1966-67 . 
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**Data obtained from individual secondary school reports to the Mississippi Accrediting Commission 
the State Accrediting Commission, 1966-67- 
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Accrediting Commission, 1966-67. 



Table IX deals with curricular offerings of accredited secondary schools. 
No attempt was made to differentiate "between secondary schools having 
three grades and those having four grades. Most senior high schools 
(grades 10-12) in the state offer those courses which are ordinarily 
applicable to the ninth grade, even though ninth grade students are not 
enrolled. This is necessary because of failures in junior high school 
and transfers from other schools. Consequently, the investigator deter- 
mined that any differences in curricular offerings between senior high 
schools (grades 10-12) and high schools (grades 9-12) would be insignifi- 
cant . 

Table XI reports data relating to special services. Because psycholog- 
ical services and public kindergartens were found to be nonexistent in 
the public schools, they were omitted from Table XI. In addition, the 
category, Special Education, represents the summary of findings from the 
survey form (see Appendix). 



II o ANALYSIS OF RESULTS 

Correlations between the independent variable, real property assess- 
ments, and the quality education factors listed in Chapter II were com- 
puted through the use of the Pearson Product-Moment formula. Correlations 
between real property assessments and two of the dependent variables, 
teachers' professional training and special services, were not obtained 
because of insufficient data in these areas. 

As indicated in Table XII, the correlations obtained are from low to 
substantial. A substantial correlation, .638, between real property 
assessments and per pupil expenditures was found. It would appear, then, 
that real property assessments best predict the per pupil expenditure 
level. Real property assessments would be a moderate predictor of 
instructional salaries, teachers 9 professional experience and curricular 
offerings and have slight relationship with library expenditures, teacher- 
pupil ratio and school dropouts. 

Further analysis indicated to the researcher that there were several 
factors which might account partially for the low level of relationship 
between real property assessments and most of the nine quality education 
factors. First, wealthy school districts do not find it necessary to 
assess real property at a high percentage of true value in order to main- 
tain schools at least as good as the average Mississippi school. On the 
other hand, less wealthy districts may be forced to assess real property 
at a higher percentage of true value in order to keep pace with wealthier 
districts. Second, some poor districts have a disproportionately high 
percentage of the total state school population, which places them in an 
extremely difficult financial position. Third, the minimum foundation 
program of education in Mississippi is designed to provide equal education 
opportunities for the youth of the state, regardless of the relative wealth 
of the individual school districts. Finally, federal funds have, in many 
districts, completely distorted the picture of local educational finance. 
For example, some extremely poor school districts spent much more for 
libraries than did other more wealthy districts. Since accrediting reports 
do not indicate how much federal money was involved, it is impossible to 
isolate these funds to get a true indication of local support. 
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TABLE XII 

PEARSON PRODUCT-MOMENT CORRELATIONS OBTAINED BETWEEN 
REAL PROPERTY ASSESSMENTS AND EACH OF SEVEN 
QUALITY EDUCATION FACTORS 



Selected Educational Factors 


Correlations with Real 
Property Assessments 


Expenditures Per Pupil 


.638 


Instructional Salaries 


.562 


Teachers* Professional Experience 


.1*31 


Libr* Expenditures 


.368 


Teacher-Pupil Ratio 


.326 


Curricular Offerings 


.1*12 


School Dropouts 


.392 
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CHAPTER IV 



CONCLUSIONS AND RECOMMENDATIONS 

I. CONCLUSIONS 

Two conclusions seem warranted as a result of the study * They are as 
follows : 

1. The relationships between real property assessment practices 
and quality education as measured by the educational factors used in 
this study range from low to substantial. 

2o The level of real property assessments is, at best, only a 
moderately valid predictor of quality education. 



II. RECOMMENDATIONS 



It is recommended that: 

1. Real property assessments be equalized throughout the state 

at a considerably higher percentage of true value than is now practiced, 
making it possible for school districts to more adequately finance public 
education. 

2. A study be made of the effects on public education of a state- 
wide real property assessment at true value, as the state constitution 
and statutes require. 

3. The level of real property assessments be considered in distribu- 
ting state education funds, rewarding districts whose assessments are at 

a higher ratio to true value and penalizing districts whose assessments 
are at a lower ratio to true value. 



APPENDIX 



2k 



o 

ERIC 



RELATIONSHIP OF REAL PROPERTY ASSESSMENT PRACTICES TO QUALITY EDUCATION 

Survey Form 



County 

School District 
School 



1-7. Blank for program 

School Identification 

8 9 10 11 12 



13. Type of school district 
I. County 

3. Municipal separate 
5o Consolidated 



Othe r 

grades included 



15. Is school accredited? 
' I. Yes 
5. No 



l4„ Type of school 

I. Elementary (grades 1-6) 

2o Elementary (grades 1-8) 

3. Junior High (grades 7-8) 

Junior High (grades 7-9) 

5. Junior-Senior High (grades 7-12) 

6. High School (grades 9-12) 

7o Senior High (grades 10-12) 
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1 6 . If accredited, by whom? 

1. State only 
5. State and regional 

IT. Independent variable (real property assessment level) 

1. Less than 5# 

2. 5# - 7# 

3 . 8 # - 10 # 

4. 11# - 13# 

5. 14# - 16# 

6. IT# - 19# 

T. 20# - 22# 

8. 23# - 25# 

9. 2 6# - 28# 

10. 29# - 31# 

18-20. For possible future use with other independent variables 
21. Expenditures per pupil 



lo 


Less than $175 


2. 


$175 - $199 


3. 


~=t 

CM 

CM 

1 

O 

o 

CM 


k. 


$225 - $2 1+9 


5. 


$250 - $27*+ 


6. 


$275 - $299 


7. 


$300 - $321+ 


8. 


$325 - $31+9 


9. 


$350 - $37*+ 


10. 


$375 and over 
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22. Instructional salaries (average) 



1 . 


Less than $3,700 


2. 


$3,700 


- $3,899 


3. 


$3,900 


- $4,099 


4. 


$4,100 


- $U ,299 


5. 


$4,300 


- $4,49 9 


6. 


$4,500 


- $4,699 


7. 


$4,700 


- $4,899 


8. 


$4,900 


- 3s;-,099 


9. 


$5,100 


- $5,299 


10. 


$5,300 


and over 



23. Teachers* professional training - f AAA* certification _____ __ 

number 



1. Less than 10 $ 

2 . 10 $ - 19 $ 

3. 20$ - 29$ 

4. 30$ - 39$ 

5. 40$ - 49$ 

6. 50$ - 59$ 

7. 60$ - 69$ 

8. 70$ - 79$ 

9. 80$ - 89$ 

10. 90$ -100$ 



24. Teachers’ professional training - ’AA' certification 



number 



1. Less than 10$ 

2. 10 % - 19 % 

3. 20 %> - 29 % 

4. 30$ - 39$ 

5. 40$ - 1*9$ 

6. 50$ - 59$ 

7. 60$ - 69$ 

80 70$ - 79$ 

9. 80$ - 89$ 

10. 90$ -100$ 



25. Teachers’ professional training - ’A’ certification 

number 



1. Less than 10$ 

2 . 10 $ - 19 $ 

3. 20$ - 29$ 

4. 30$ - 39$ 

5. 40$ - 49$ 

6. 50$ - 59$ 

7. 60$ - 69$ 

8. 70$ - 79$ 

9. 80$ - 89$ 

10. 90$ -100$ 



26o Teachers* professional training - f B* certification 



number 



1. 


Less than 10# 


2 0 


10# 


- 19% 


3o 


20# 


- 29 % 


k. 


30 % 


- 19 % 


5o 


ko% 


- 49% 


6o 


30 % 


- 59 % 


7. 


6 o% 


ON 

VO 

1 


8o 


10 % 


- 19 % 


9o 


CO 

O 


- 89% 


10 0 


VO 

o 


-100% 



27 * Teachers* professional training - *C f certification 



1. 


Less 


than 10% 


2 0 


10# 


- 19% 


3o 


O 

CM 


- 29% 




30% 


- 39% 


5o 


4o% 


- 49% 


6. 


50% 


- 59% 


7. 


o 

VO 


- 69% 


8e 


70% 


- 79% 


9o 


o 

oo 


- 89% 


10 0 


VO 

o 


-100% 



number 



29 



28, Teachers’ professional training - 'D* certification 



number 



lc 


Less 


than 10? 


2. 


10# 


- 19 % 


3. 


20# 


- 29% 


4. 


30 % 


- 39% 


5* 


ko% 


- k9% 


6, 


30 % 


- 39% 


7. 


60 $ 


- 69 % 


8, 


70 % 


- 79% 


9° 


o 

CO 


ON 

CO 

1 


10. 


90# 


-100# 



29 o Teachers * professional training - *E* certification 



lo 


Less than 10# 


2 o 


10# - 19# 


3o 


20# - 29# 


4o 


30# - 39# 


5o 


ON 

1 

O 


6„ 


50# - 59# 


? 0 


60# - 69# 


8o 


TO# - T9# 


9o 


CO 

0 

1 

oo 

vo 


10 0 


90# -100# 



number 



30 




30 o Teachers 1 professional experience (average) 



lo 


Less than 5 years 


2. 


5 - 6 


years 


3. 


T - 8 


years 


4o 


9 -10 


years 


5o 


11-12 


years 


6. 


13-1*+ 


years 


To 


15-16 


years 


8o 


17-18 


years 


9o 


19-20 


years 


10 o 


21 years and over 



31 o Library expenditures per pupil 



lo 


Less than $1.00 


2c 


$lo00 


- $1.99 


3o 


$2o00 


- $2.99 


1+0 


$3o00 


- $3.99 


5o 


$4o00 


- $4.99 


6o 


$5o00 


- $5.99 


7o 


$6o00 


- $6.99 


8o 


$7o00 


- $7.99 


9o 


$8o00 


- $8.99 


10 o 


$9o00 


and over 



32. Teacher-pupil ratio 



1. Less than 20 

2 . 20 - 21 

3. 22 - 23 

4. 24 - 25 

5. 26 - 27 

6 . 28 - 29 

7. 30 - 31 

8. 32 - 33 

9. 34 - 35 

10. 36 and over 



33. Curricular offerings (high school units) 

1. Less than 20 

2. 20 - 23 

3. 24 - 27 

4o 28 - 31 

5. 32 - 35 

6. 36 - 3 ? 

7. 40 - 43 

8. 44 - 47 

9. 48 - 51 

10. 52 and over 
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o 

ERLC 



34. School dropouts (1966-67) 



lo Less than 1 % 

2. 1.035 - 1.4935 

3. 1.535 - 1.9935 

4. 2.035 - 2.4935 

5. 2.535 - 2.9935 

6. 3.035 - 3.49$ 

7. 3.535 - 3.9935 

8. 4.035 - 4.49% 

9. 4.535 - 4.99$ 

10. 535 and over 

35 • Guidance services 

1. Yes 

5. No 

36. If yes, ratio of guidance personnel to 
1. 1:100 

2. 1:200 

3. 1:300 

4. 1:400 

5. 1:500 

6. 1:600 

7. 1:700 

8. 1:800 

9. 1:900 

10 o 1:1,000 and over 
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o 

ERLC 



Enrollment 



pupils 



Vocal music program 
1. Yes 
5. No 

Instrumental music program 
1. Yes 
5* No 

Psychological services 
lo Yes 

5. No 

School health program (school nurse) 
lo Yes 
5. No 



Physical education program 
lo Yes 
5. No 

Art program 

lo Yes 

5. No 
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Foreign language program 
1 o Yes 
5. No 

Summer school program 
1. Yes 
5» No 

Special education program (mental retardation) 

1. Yes 
5» No 

Special education program (exceptional children) 
lo Yes 
5 c No 

Special education program (physically handicapped) 
lo Yes 
5o No 

Public kindergarten 
lo Yes 
5o No 
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